Fixed-Delay Events in Generalized Semi-Markov Processes Revisited
نویسندگان
چکیده
We study long run average behavior of generalized semi-Markov processes with both fixed-delay events as well as variable-delay events. We show that allowing two fixed-delay events and one variable-delay event may cause an unstable behavior of a GSMP. In particular, we show that a frequency of a given state may not be defined for almost all runs (or more generally, an invariant measure may not exist). We use this observation to disprove several results from literature. Next we study GSMP with at most one fixed-delay event combined with an arbitrary number of variable-delay events. We prove that such a GSMP always possesses an invariant measure which means that the frequencies of states are always well defined and we provide algorithms for approximation of these frequencies. Additionally, we show that the positive results remain valid even if we allow an arbitrary number of reasonably restricted fixed-delay events.
منابع مشابه
Solving Generalized Semi-Markov Processes using Continuous Phase-Type Distributions
We introduce the generalized semi-Markov decision process (GSMDP) as an extension of continuous-time MDPs and semi-Markov decision processes (SMDPs) for modeling stochastic decision processes with asynchronous events and actions. Using phase-type distributions and uniformization, we show how an arbitrary GSMDP can be approximated by a discrete-time MDP, which can then be solved using existing M...
متن کاملA Formalism for Stochastic Decision Processes with Asynchronous Events
We present the generalized semi-Markov decision process (GSMDP) as a natural model for stochastic decision processes with asynchronous events in hope to spur interest in asynchronous models, often overlooked in AI literature.
متن کاملSolving Generalized Semi-Markov Decision Processes Using Continuous Phase-Type Distributions
We introduce the generalized semi-Markov decision process (GSMDP) as an extension of continuous-time MDPs and semi-Markov decision processes (SMDPs) for modeling stochastic decision processes with asynchronous events and actions. Using phase-type distributions and uniformization, we show how an arbitrary GSMDP can be approximated by a discrete-time MDP, which can then be solved using existing M...
متن کاملNumerical Analysis of Generalized Semi-Markov Processes
This paper present methodological results that allow the cost-effective numerical analysis of finite-state generalized semi-Markov processes (GSMPs) with exponential and deterministic events by an embedded general state space Markov chain (GSSMC). Key contributions constitute the formal proof that elements of the transition kernel of the GSSMC can always be computed by appropriate summation of ...
متن کامل